Measuring The VC - dimension Using OptimizedExperimental

VC-dimension is the measure of model complexity (capacity) used in VC-theory. The knowledge of the VC-dimension of an estima-2 tor is necessary for rigorous complexity control using analytic VC generalization bounds. Unfortunately, it is not possible to obtain the analytic estimates of the VC-dimension in most cases. Hence, it has been recently proposed to measure the VC-dimension of an estimator experimentally by tting the theoretical formula to a set of experimental measurements of the frequency of errors on arti-cially generated data sets of varying sizes (Vapnik et al, 1994). However, it may be diicult to obtain an accurate estimate of the VC-dimension due to variability of random samples in the experimental procedure proposed by (Vapnik et al, 1994). We address this problem by proposing an improved design procedure for specifying the measurement points (i.e., the sample size and the number of repeated experiments at a given sample size). Our approach leads to non-uniform design structure as opposed to the uniform design structure used in the original paper (Vapnik et al, 1994). Our simulation results show that the proposed optimized design structure leads to a more accurate estimation of the VC-dimension via the experimental procedure. Likewise, our results show that more accurate estimation of VC-dimension leads to improved complexity control via analytic VC-generalization bounds and hence better prediction accuracy.